Multivariate count time series segmentation with "sums and shares” and Poisson lognormal mixture models: a comparative study using pedestrian flows within a multimodal transport hub

This paper deals with a clustering approach based on mixture models to analyze multidimensional mobility count time-series data within a multimodal transport hub. These time series are very likely to evolve depending on various periods characterized by strikes, maintenance works, or health measures against the Covid19 pandemic. In addition, exogenous one-off factors, such as concerts and transport disruptions, can also impact mobility. Our approach flexibly detects time segments within which the very noisy count data is synthesized into regular spatio-temporal mobility profiles. At the upper level of the modeling, evolving mixing weights are designed to detect segments properly. At the lower level, segment-specific count regression models take into account correlations between series and overdispersion as well as the impact of exogenous factors. For this purpose, we set up and compare two promising strategies that can address this issue, namely the "sums and shares” and "Poisson log-normal” models. The proposed methodologies are applied to actual data collected within a multimodal transport hub in the Paris region. Ticketing logs and pedestrian counts provided by stereo cameras are considered here. Experiments are carried out to show the ability of the statistical models to highlight mobility patterns within the transport hub. One model is chosen based on its ability to detect the most continuous segments possible while fitting the count time series well. An in-depth analysis of the time segmentation, mobility patterns, and impact of exogenous factors obtained with the chosen model is finally performed. © 2023, Springer-Verlag GmbH Germany, part of Springer Nature.

An incremental loss ratio method using prior information on calendar year effects.

In a run-off triangle external factors can have a similar influence on all incremental losses of the same calendar year. This can distort the triangle such that reserving methods like chain ladder or the loss ratio method do not work properly. A very recent example of such an external factor is the Covid-19 pandemic. In many countries, the insurance industry is in the process of establishing market knowledge about the impact of the pandemic on premiums and losses. We extend the additive claims reserving model to allow for calendar year effects and develop a variant of the incremental loss ratio method (also known as the additive method) that can make use of such market knowledge. We derive formulas for the mean squared error of prediction and provide a detailed numerical example. Supplementary Information: The online version contains supplementary material available at 10.1007/s13385-022-00315-3.

COVID-19 Dataset Clustering based on K-Means and EM Algorithms

In this paper, a COVID-19 dataset is analyzed using a combination of K-Means and Expectation-Maximization (EM) algorithms to cluster the data. The purpose of this method is to gain insight into and interpret the various components of the data. The study focuses on tracking the evolution of confirmed, death, and recovered cases from March to October 2020, using a two-dimensional dataset approach. K-Means is used to group the data into three categories: "Confirmed-Recovered”, "Confirmed-Death”, and "Recovered-Death”, and each category is modeled using a bivariate Gaussian density. The optimal value for k, which represents the number of groups, is determined using the Elbow method. The results indicate that the clusters generated by K-Means provide limited information, whereas the EM algorithm reveals the correlation between "Confirmed-Recovered”, "Confirmed-Death”, and "Recovered-Death”. The advantages of using the EM algorithm include stability in computation and improved clustering through the Gaussian Mixture Model (GMM). © 2023,International Journal of Advanced Computer Science and Applications. All Rights Reserved.

Robust autoregressive modeling and its diagnostic analytics with a COVID-19 related application

Autoregressive models in time series are useful in various areas. In this article, we propose a skew-t autoregressive model. We estimate its parameters using the expectation-maximization (EM) method and develop the influence methodology based on local perturbations for its validation. We obtain the normal curvatures for four perturbation strategies to identify influential observations, and then to assess their performance through Monte Carlo simulations. An example of financial data analysis is presented to study daily log-returns for Brent crude futures and investigate possible impact by the COVID-19 pandemic. © 2023 Informa UK Limited, trading as Taylor & Francis Group.

Using Random Effect Models to Produce Robust Estimates of Death Rates in COVID-19 Data.

Tracking the progress of an infectious disease is critical during a pandemic. However, the incubation period, diagnosis, and treatment most often cause uncertainties in the reporting of both cases and deaths, leading in turn to unreliable death rates. Moreover, even if the reported counts were accurate, the "crude" estimates of death rates which simply divide country-wise reported deaths by case numbers may still be poor or even non-computable in the presence of small (or zero) counts. We present a novel methodological contribution which describes the problem of analyzing COVID-19 data by two nested Poisson models: (i) an "upper model" for the cases infected by COVID-19 with an offset of population size, and (ii) a "lower" model for deaths of COVID-19 with the cases infected by COVID-19 as an offset, each equipped with their own random effect. This approach generates robustness in both the numerator as well as the denominator of the estimated death rates to the presence of small or zero counts, by "borrowing" information from other countries in the overall dataset, and guarantees positivity of both the numerator and denominator. The estimation will be carried out through non-parametric maximum likelihood which approximates the random effect distribution through a discrete mixture. An added advantage of this approach is that it allows for the detection of latent subpopulations or subgroups of countries sharing similar behavior in terms of their death rates.

An Expectation-Maximization Algorithm for Including Oncological COVID-19 Deaths in Survival Analysis.

We address the problem of how COVID-19 deaths observed in an oncology clinical trial can be consistently taken into account in typical survival estimates. We refer to oncological patients since there is empirical evidence of strong correlation between COVID-19 and cancer deaths, which implies that COVID-19 deaths cannot be treated simply as non-informative censoring, a property usually required by the classical survival estimators. We consider the problem in the framework of the widely used Kaplan-Meier (KM) estimator. Through a counterfactual approach, an algorithmic method is developed allowing to include COVID-19 deaths in the observed data by mean-imputation. The procedure can be seen in the class of the Expectation-Maximization (EM) algorithms and will be referred to as Covid-Death Mean-Imputation (CoDMI) algorithm. We discuss the CoDMI underlying assumptions and the convergence issue. The algorithm provides a completed lifetime data set, where each Covid-death time is replaced by a point estimate of the corresponding virtual lifetime. This complete data set is naturally equipped with the corresponding KM survival function estimate and all available statistical tools can be applied to these data. However, mean-imputation requires an increased variance of the estimates. We then propose a natural extension of the classical Greenwood's formula, thus obtaining expanded confidence intervals for the survival function estimate. To illustrate how the algorithm works, CoDMI is applied to real medical data extended by the addition of artificial Covid-death observations. The results are compared with the estimates provided by the two naïve approaches which count COVID-19 deaths as censoring or as deaths by the disease under study. In order to evaluate the predictive performances of CoDMI an extensive simulation study is carried out. The results indicate that in the simulated scenarios CoDMI is roughly unbiased and outperforms the estimates obtained by the naïve approaches. A user-friendly version of CoDMI programmed in R is freely available.

Timely epidemic monitoring in the presence of reporting delays: anticipating the COVID-19 surge in New York City, September 2020.

BACKGROUND: During a fast-moving epidemic, timely monitoring of case counts and other key indicators of disease spread is critical to an effective public policy response. METHODS: We describe a nonparametric statistical method, originally applied to the reporting of AIDS cases in the 1980s, to estimate the distribution of reporting delays of confirmed COVID-19 cases in New York City during the late summer and early fall of 2020. RESULTS: During August 15-September 26, the estimated mean delay in reporting was 3.3 days, with 87% of cases reported by 5 days from diagnosis. Relying upon the estimated reporting-delay distribution, we projected COVID-19 incidence during the most recent 3 weeks as if each case had instead been reported on the same day that the underlying diagnostic test had been performed. Applying our delay-corrected estimates to case counts reported as of September 26, we projected a surge in new diagnoses that had already occurred but had yet to be reported. Our projections were consistent with counts of confirmed cases subsequently reported by November 7. CONCLUSION: The projected estimate of recently diagnosed cases could have had an impact on timely policy decisions to tighten social distancing measures. While the recent advent of widespread rapid antigen testing has changed the diagnostic testing landscape considerably, delays in public reporting of SARS-CoV-2 case counts remain an important barrier to effective public health policy.

Finite mixture model of hidden Markov regression with covariate dependence

In recent days, a combination of finite mixture model (FMM) and hidden Markov model (HMM) is becoming popular for partitioning heterogeneous temporal data into homogeneous groups (clusters) with homogeneous time points (regimes). The regression mixtures commonly considered in this approach can also accommodate for covariates present in data. The classical fixed covariate approach, however, may not always serve as a reasonable assumption as it is incapable of accounting for the contribution of covariates in cluster formation. This paper introduces a novel approach for detecting clusters and regimes in time series data in the presence of random covariates. The computational challenges related to the proposed model has been discussed, and several simulation studies are performed. An application to United States COVID-19 data yields meaningful clusters and regimes.

A state-space approach to time-varying reduced-rank regression

We propose a new approach to reduced-rank regression that allows for time-variation in the regression coefficients. The Kalman filter based estimation allows for usage of standard methods and easy implementation of our procedure. The EM-algorithm ensures convergence to a local maximum of the likelihood. Our estimation approach in time-varying reduced-rank regression performs well in simulations, with amplified competitive advantage in time series that experience large structural changes. We illustrate the performance of our approach with a simulation study and two applications to stock index and Covid-19 case data. [ FROM AUTHOR] Copyright of Econometric Reviews is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)

Multiple change point clustering of count processes with application to California COVID data

In this paper, a model-based clustering algorithm relying on a finite mixture of negative binomial Lévy processes is proposed. The algorithm models heterogeneous stochastic count process data and automatically estimates multiple change points upon fitting the mixture model. Such change point estimation identifies time points when deviation from the standard process has occurred and serves as an important diagnostic tool for analyzing temporal data. The proposed model is applied to the COVID-positive ICU cases in the state of California with very interesting results.

Regression Models for Understanding COVID-19 Epidemic Dynamics With Incomplete Data.

Modeling infectious disease dynamics has been critical throughout the COVID-19 pandemic. Of particular interest are the incidence, prevalence, and effective reproductive number (Rt). Estimating these quantities is challenging due to under-ascertainment, unreliable reporting, and time lags between infection, onset, and testing. We propose a Multilevel Epidemic Regression Model to Account for Incomplete Data (MERMAID) to jointly estimate Rt, ascertainment rates, incidence, and prevalence over time in one or multiple regions. Specifically, MERMAID allows for a flexible regression model of Rt that can incorporate geographic and time-varying covariates. To account for under-ascertainment, we (a) model the ascertainment probability over time as a function of testing metrics and (b) jointly model data on confirmed infections and population-based serological surveys. To account for delays between infection, onset, and reporting, we model stochastic lag times as missing data, and develop an EM algorithm to estimate the model parameters. We evaluate the performance of MERMAID in simulation studies, and assess its robustness by conducting sensitivity analyses in a range of scenarios of model misspecifications. We apply the proposed method to analyze COVID-19 daily confirmed infection counts, PCR testing data, and serological survey data across the United States. Based on our model, we estimate an overall COVID-19 prevalence of 12.5% (ranging from 2.4% in Maine to 20.2% in New York) and an overall ascertainment rate of 45.5% (ranging from 22.5% in New York to 81.3% in Rhode Island) in the United States from March to December 2020. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.